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From an extensive study involving 2,877 senior education majors in 86 
teacher training institutions, Mayo (1967i) concluded that beginning teachers do 
iTiot possess a high degree of knowledge or skill in educational measurement, and 
ithat they show only small gains in competency after two years of classroom 
i Inexperience. Goslin (1967) emphasizes the importance of adequate training in 

measurement, stressing the large aiaount of standardized testing done in £:chools. 
Teacher-m.ade tests are used even more frequently but are often poor in quality 
(Stianley and Hopkins, 1972). Glaser (1973) suggests that testing could become 
an even more integral part of the educational process in the future, with a shift 
to criterion-referenced tests and a continual matching of student capabilities 
to educational options^ The problem of teacher competency in measurement is a 
critical one. 

It has been suggested (e.g., Gos'lin, 1957a; Mayo, 1967, 1970a) that attitude 
toward educational measurement may be an obstacle to achievement in course work 
■» or to later implementation of measurement principles. Mayo (1967) proposes that 
many students may be hindered by (a) a negative attitude toward statistics-- 
statistics may be soen as irrelevant to most teachers' needs--and by (b) a more 
general failure, in spite of course ^./ork, to appreciate the usefulness of 
measurement principles, with a resulting (c) conceptual and affective gap betwccA 
teachers and students of measurcnient. Possible positive and negative consequences 
of testing have been noted (e.g., Bloom, 19685 Goslin, 1967b). Some students maybe 
focu<->ing on perceived negative implications of present measurement practice, 
linking measuiremcnt with a variety of assumed educational and social ills (Schutz, 
1971). Indeed, many recent writings of educational reform vicvj testing as a 
"central c^^vil to be challenged and exorcised (Glaser, 1973, p. 564).*' Such 
writings 'r.ay both reflect and influence student attitude toward educational measure- 
ment and, pcrha]-s, toward instructors of measurement. 

Preliminary to constructing the test used in his survey of teachers^ 
measurement competency, Mayo (1967) surveyed a variety of measurement specialists 
and educator-ion the importance of 70 measurement objectives; these objectives w r:.- 
developed trom a content outline of an NCME committee. Most objectives were seen as 
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Important by this rj;->iple, with those relating to classroom tests most important 
in general, followed by objectives involving standardized tests, uses of measurement 
and evaluation, and statist fcs. Measurement students' priorities may be somewhat 
different. There is a need for a similar survey of attitudes of the beginning 
measurement student tc determine importance, and relative importance, of specific 
measurement objectives. Meaningfulness and relevance of less attractive material 
could be stressed, with possible gains in both attitude and mastery (Mayo, 1967, 
1970a, 1970b). Relationships between measurement objectives perceived as less 
and more important might be emphasized, in particular. 



Methods 

For the present sti'xW, a list of 47 measurement objectives was developed. 
Items were chosen to relate, at a more general level, to those included in the 
Mayo list, and to cover possible measurement topics not included by Mayo (e.g», 
criterion-referenced tests, affective objectives, historical precedents of present 
testing practice). A few items were suggested by recurring requests of students 
in measurement classes and related suggestions in the literature (e.g.. Mayo, 1970a, 
1970b) --for example, to develop ways to minimize test anxiety or let students 
have input in defining classroom objectives. Most objectives involved activities 
at or above the application level of Bloom et al^s (1956) cognitive taxonomy. 
Those items involving specific uses of tests were presented as evaluation level 
objectives and were supplemented by several affective items which involved defining 
one's own position on measurement alterna\:ives; it was fell that some students might 
consider these items important whether positive or negative toward current measurement 
practice. 

Items were written in non- technical language, or with technical terms defined, 

where included, and were presented in an order which was randomly determined. The 

general introductory stem, "A coui ">e in educational measurement can enable me to:" 

was included at t:ho top of each page. Subjects v/ere asked t'^ rate each objective 

for importance, using a five point scale which ranged from *Very important*' to 

"very unir.ipor tan t . " ^ A further response option ("X*') was provided for those who 

did not understand the statement. Students were asked to put down a number to 
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identify their papers, but were not asked to give their names and were assured of 
the confidentiality of their answers. They were told the rosearch.^r was interested 
in student attitudes toward aspects of an educational measurement course and were 
encouraged to respond as carefully and candidly as possible. 

Subjects were 23 4 undergraduates in education who had nov yet taken a measure- 
i^Pi^^"it course.^ One-hundred and eighty students were in secondary education, and 54 
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were in elementary education. Most secondary students were administered the 
instrument as the initial activity in their educational measurement course. 
Elementary students beginning measurement could not be included (as naive 
subjects) as they were already involved in another study. Instead, the 
instrument was given to several educational psychology sections of a required 
education core course, typically taken by sophomores, the year before measure- 
ment. The instrument was administered by the author to all but about 15 of the 
subjects. 



Results and Discussion 

The typical objective was understandable to almost all of the r espoiidents. 

Ratings for only one item (#46, correlation coefficient) had to be interpreted 
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cautiously, due to a high proportion of "X'' responses. There was considerable 
similarity between responses of the 27 secondary students in the educational 
psychology classes and the remaining secondary sample.^ On the assumption that 
elementary responses would also be similar aero.:: contexts, the elementary sample 
was also included for comparison In the data analysis. 

The sections below address: (a) overall rank ordering of objectives by 
importance (b) differences between males and females, and students in elementary 
and secondary education, and (c) factor analytic structure of objectives. 

Rank Order of Ob|ectives by Importance 

Objectives ranked by median importance for the entire sample are presented 
in Table 1. The median was employed dae to distributional skew. The Spearman 
correlation between this list and rankings from the smaller elementary subsaraple 
is 0.93; thus, priorities of botli elementary and secondary groups are reflected 
in this list. Median ratings ranged from 1.19 (very important) down to 2.94 
undecided). In general, no items were felt to be unimporuant, and ratings were 
generally skewed toward positive importance. However, variation was considerable 
for all objectives, with the full range of attitude present in almost every case. 
(The typical standard deviation was about 1.0--and standard error of the mean 
about 0.07--for these items). 

In gen./'al, items rated highest in importance involved critical analysis of or 
alternative approaches to measurement procedures; the least favored dealt with 
statistics. Items involving similar general content were not always considered 
i^Pj^^-tilar in importance. For example, items involving objectives were primarily 
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high in importance, while those involving test construction and standardized 
tests, although lower in general, were more varied in importance. Items on 
reliability and validity were somewhat varied but generally high. Items fiddressed 
to uses of tests were rated high in importance in the context of feedback and 
goal setting, and improving curriculum and instruction, but were lower when 
applied to placement and grouping, and grading.^ 

Results suggest that students are: (a) positive or somewhat positive in 
general to the importance of most testing and measurement procedures, with perceived 
importance varying as a function of the types of procedures involved and their 
intended use, and (b) undecided, overall, as to the importance of statistics. This 
was a somewhat more complex ordering of priorities than that of Mayors group of 
educators. Students aloO seemed (c) particularly concerned with evaluating measuring 
instruments and procedures and deciding on their own use of testing methods* 

It is notable that many of the objectives perceived as most important involve 
higher level cognitive or affective goals. A large number relate to the synthesis 
and, particularly, evaluation levels of Bloom et al^s (1956) cognitive taxonomy; 
many less important objectives are lower in taxonomy level. Interestingly, many 
of the lower-ranked--and, often, lower- taHonomy--objectives are useful and even 
required to address some of those objectives considered most important. Use of 
statistical methods in estimating instrument reliability and validity is one 
notable example. These are connections which need to be stressed in instruction. 

Differences Bctv7cen Groups 

Elementary Females ^ Sec -ndary Males and Females . While there were many 
more similarities than differences between subgroups of this sample, some of 
the differences are of interest. Table 2 includes results from significant univariate 
analyses of variance and Dunn multiple comparisons between means, following a 
significant multivariate F-ratio, for 48 elementary females, 120 secondary females, 
and 58 secondary males. (Subjects who didn't report their sex were eliminated 
as were the few males in he elementary education group.) Several significance 
levels need to be interpreted with caution, due to inliomogenei ty of variance.^ 
General results will be emphasized here. 

All but two significant differences u i^^* ir: ;c secondary males and females-- 
and, in some cases, elementary females as well. Females were more positive than 
males toward cTxr.Vx tUioSv* item:, a trend which was generally borne out 
in the non- signi f ican t items. Typically, means for elementary and secondary 
females were similar, such that a larger elementary sample might have caused these 
variables to distinguish males and females generally, across groups. Di £ cingui shing 
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items involved diverse content (test usage, item-writing, statistics). A number 
of the most distinguishing variables involved relatively less popular aspects of 
measurement. 

Results suggest more differences between males and females than between 
elementary and secondary students generally. This may partially reflect the fact 
that elementary and secondary students have had essentially the same course experience 
in education p:ior to the year of measurement. Beyond this, (secondary) females may 
be somewhat more receptive than (secondary) males, in general, to aspects of 
the curriculum. (Results might also reflect the fact that a female typically 
administered the instrument to these subjects.) 

Area of Spacialization . Information on area of specialization was available 
on some secondary subjects. These subjects were divided into groups of males and 
females in humanities /social science (English, language, social studies education) 
and math/sciences (mathema:ic s , science, business education), yielding 26 males and 
51 females in humanities/social science (H/SS) and 15 males and 26 females in math/ 
sciences (M/S). Most of the subjects were in English and social studies, and math 
and business. 

Multivariate analysis of variance showed significant main effects on both sex 
and specialty; there was no significant interaction. Most of the sex differences 
in the previous analysis reappeared for this sample. The second part of Table 2 
gives the significant differences between H/SS and M/S groups. 

ll/S students gave more importance than H/SS students to constructing good 
objective items and evaluating test uses in grading. H/SS students were more 
positive toward the re-^ainder of the items. Greatest differences involved 
constructing good essay questions and recognizing situations in which norms may be 
useful or misleading in score interpretation (both p<.0001). Remaining differences 
involved a variety of items (testing atmosphere, standardized tests and norms, 
testing programs, historical a«^pccts, statistics). 

H/SS and M/S students differ as one might expect on types of item writing 
considered important, suggesting different instructional emphases for these 
groups. The more general positive disposition of H/SS may suggest relatively higher 
interest in understanding and making one's own deci s ions cL bout selec ted aspects 
of testing. It does not necessarily suggest more positive disposition toward present 
measurement practice. 



Fac tor Ati -^'Z'^- - 

A factor analysis was performed on data from the entire sample. A principal 
ip'iponents solution produced 13 factors with eigenvalues greater than one. 
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accounting for 62.6% o£ the variance. These factors were rotated to the normal 

varitnax criterion. Results are presented in Table 3. The present data didn't 

lend itself readily to simple structure. Beyond this, factor identity was often 

unclear; factors did not necessarily include items with similar content. (Some 

9 

factors can, however, be tentatively identified. ) 

The present analysi' indicates that relationships between some variablesare 
complex. Items involving, for exauiple, objectives, standardized tests, and even 
statistics, are somewhat dispersed among factors, with factors such as 1 defined 
by a seemingly heterogeneous group of items. This may indicate that students 
see similarities between some objectives as a complicated function of type of 
procedure, purpose, and context of use. It may alternatively indicate a lack of 
conceptual clarity about the curriculum. In some cases (e.g.. Factor VIII), 
objectives of similar content load on the same factor but are notably different 
in perceived importance, perhaps indicating a perceived differential effectiveness 
in addressing the same problem. Instruction might create different relationships 
between measurement ob jec tives and greater similarity in intra-f actor item importance, 
as students see new applications and relevance(s) of various measurement procedures. 

Conclusions 

While there were a few differences, subgroups in this sample felt similar in 
general toward topics in educational measurement; overall, the present sample seemed 
not to be negatively disposed toward the curriculum. Students were generally un- 
decided about the importance of statistics, and varied in attitude toward the 
importance of other objec tives^ results somewhat different than those from the Mayo 
(1967) sample of educators. Also, there was notable latudent interest: in addressing 
certain measurement issues, and in evaluating certain measurement procedures, both 
in general and in the context of their own future teaching situations. 

Findings are contrary to Mayors suggestion of a general negative disposition 
toward measurement, \^^lile this sample could be different, Mayo's general 
observations would seem to apply to Boston University. Perhaps a vocal minority 
makes itself well heard; indeed, distribution of response was broad on most 
it em 5^ and some respondents were very negative on almost every item on the instrument. 
Or, perhaps^ negative attitude is developed or increased during instruction. Mayo 
(1967, 1970a) suggested that the relevance of some measurement material may not be 
adequately coinmunicated. The present study suggests that for the entering student, 
the full relevance of some measurement material to high-priority student objectives 
has yet to be established. Perhaps, in addition, material students feel is most 
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Important is not always stressed in instruction, and student expectations for the 
course are not realized. Negative attitude could certainly increase in this event. 

Instruction in measurement could focus on existing student concerns and relate 
material considered less important to these- -always emphasizing its relevance and 
importance. Mayo (1970a) reports some innovative teaching techniques which 
appear promising. With such an emphasis, undergraduates in education might come 
to more fully realize the relevance of measurement principles and te^nniques to 
their future goals as teachers. 
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Footnotes 

^Subjects were also asked to rate these objectives £or interest. Interest and 
importance ratings were strongly related. Only importance ratings will be 
discussed here. 

2 

In about half the cases, this was the last five digits of the student* s 
home phone number and, in the remainder, their social security number. 

3 

One group of subjects was dropped from the origin<?.l sample due to non-standard 
administration of the instrument. Students in early childhood and special education 
were also excluded from the present analyses. 

4 

There were 75 "X" responses (32%) to this item. Otherwise, '^X" responses 
ranged from 0 to 34 with a median close to 5 and a semi-interquartile range of 
approximately 4. 

^The Spearman rank order correlation between importance ratings of secondary 
education sophomores and juniors was .89. Rank order correlations between elementary 
students' ratings and secondary sophomores' and juniors' ratings were .89 and .90, 
respectively. (Reliability of the overall rank ordering is also implied in these 
results.) Multivariate analysis of variance showed no significant differences 
between the two secondary groups* 

more detailed breakdown follows. Particularly notable are inconsistencies 
in ranking, where "less important'' objectives are strongly related or prerequisite 
to more highly placed ones. 

Three highly rated items involved some aspect of reliability or validity (ranks 
of 3,6,9), while two other such items (18,28) were further down the list. 
Evaluation of test usage for improving curriculum & instruction and for feedback and 
goal setting (5,10) were also very high, while evaluation of uso.q- "f^r rloCi/.^/.-.u*: 
and grouping and for grading (30,32) were less important. Compared to #32, descrip- 
tion of reasons for, and systems of, grading (14) emerged relatively more important. 

Uses of objectives were generally rated high with teacher-student definition 
of objectives, affective objectives, objectives in planning, and different level, 
and cognitive objectives (2,9.11,13,15) particularly high. Use of behavioral 
objectives (25) was seen of lesser importance. 

Some interesting juxtapositions involved teacher-made tests. Defining a 
position on when to use teacher- tests in one's classroom (12) was rated fairly high 
in importance, while various items relating to the actual designing of measures 
(20,26) including construction of essay and objective items (21,27) were in the 
middle, or somewhat important, range. Objectives involving item analysis and 
anecdotal records (33,36) were a little lower. Objectives dealing with semi- 
objective items and rating scales (40,47) were near the bottom and were clearly less 
popular than most other ty}:)es of measures. Of greatest importance was being able 
to devise new types of instruments v/here required (4). Being able to contrast 
different types of measures (23,29), including teacher-made and s*. i ' dardized tests, 
was seen as somewhat important. 

Some other interesting juxtapositions involved standardized tests* Defining 
a position on one's own usage (8) was seen as very important. However, items 
relating to specific test-types (22,25,34) v7ere seen as less important as ^'/crc-- 
and more strongly so--items involving school and district testing programs (38) and 
standard reference sources (41). Proper administration and scoring (17) was 
considered fairly important. Appropriate use of norms (19) was rated somewhat high 
while being able to describe their characteristics and uses (37) was rated much 
- - er. 
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Defining a position on usage o£ criterion- and norm-referenced tests in the 
schools (31) was rated somewhat important, while understanding their current use 
(39,43) was substantially lower. Four items involving statistics (42,44,45,46)-- 
distributions, central cendency and variability, and correlation--appeared at the 
bottom of the list. (One item involving correlation (28) was higher where applied 
to test reliability and validity.) Two other items involved ethical and historical 
aspects o£ testing (16,35) with the first seen as fairly important, and relatively 
more important than the second. The most highly rated item involved making test- 
taking more enjoyable (1). 

''some distributions showed marked skewing. More important here is occasional 
inliomogenei ty of variance, given unequal cell n ' s (Glass, Peckham, and Sandeis, 1972). 
In particjlar, alpha values for variables 7 & 8 in the first analysis anf^ variables 
1 & 7 in the second analysis, may be higher than reported. Some other effects may 
be more significant than indicated in this analysis. 

An orthoblique solution using Kaiser^s (in press) Litnie Jiffy routine 
produced no essential improvement; it did, however, produce an overall "index of 
factorial simplicity'' of 0.66 (poor). A direct oblirin procedure was unable to 
reach convergence in a substantial number of iter luions* 

9 

The first three factors were par ticv^ dr ly strong, relative to the others. 
Factor I is hard to identify; both its ^'efining and related items involve types of 
student-oriented test usage (32,30, "^0;, score interpretation and usage (42,43), 
item/instrument construction (47,27,40), and behaviors related to standardized 
tests (41jl7,24). (Note that while some loadings are relatively low, some 
communalities are likewis^: low. Many of the items of extremely low communality 
showed some restriction of range and might have contributed more powerfully 
otherwise. ) 

One defining item also involved objectives in classroom planning (11). The 
strongest loading item involved test usage in grading (32). Items varied in 
importance, although many were rated relatively low. Perhaps this factor relates 
to some global aspect of perceived classroom procedures. 

Factor II is defined primarily by several aspects of statistics, all rated 
relatively low. Factor III involves items, of varying importance, which seem to 
suggest a broader view of the context in which measurement takes place. 

Among the weaker factors. Factor IV is curious ly defined by two evaluative 
items involving uses of tests to improve curriculum and instruction (5) and 
estimation of iristrument validity (6). All relevant items are fairly high in 
importance. Factor V is loaded on most strongly by making testing more enjoyable 
( 1) and, interestingly, is also defined by construction of essay quest ion-. (21) , 
and use of item analysis (33) --objectives of varying importance. Factor VI is 
related to use of behavioral objectives (25), Factor VII to criterion-referenced 
tests (31,39), Factor VIII to aspects of reliability and validity (28,9,18), Factors, 
X and XI to special aspects of standardized testing (7,22) and objectives (15,13)^ 
and Factor XII is related to devising one's own cognitive and affective measures 
(26,20). 

While these factors cannot be considered to be stable, or the solution 
unique, this analysis provides some indication of the complex basis on which students 
are making distinctions between items. Objectives loading on a factor frequently 
don't show patterns in content, or taxonomy or importance level. 
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Table 1 



Median Importance Ratings of 47 Measurement Objectives 
for 234 Students in Elementary and Secondary Education 



Objectives^ 

!• Devise procedures, make test-taking more enjoyable, my students 

2. Devise procedures, teacher-student definition of objectives 

3# Distinguish "good'* & ''bad" tests, usage, testing practice today 

4. Devise new types instruments to meet my own needs 

5. Evaluate uses of tests to improve curriculum, instriiction 
6» Decide if instrument measuring what it^s supposed to 

7. Define own position, standardized test uses, my students 

8. Employ educational objectives, affective goals 

9. Estimate how consistently a test is doing its job 

10. Evaluate uses of tests, inform progress, difficulties, set goals 

Employ defined educational objectives, planning class activities 
Define own position, teacher-made test uses, my classroom 
Employ educational objectives, different levels complexity 
Describe reasons for, systems of, grading 
Employ educational objectives, cognitive goals 
Define own position, ethical responsibilities of tester 
Employ appropriate procedures, administering & scoring tests 
Estimate error in measurements on students 

Recognize situations where norms useful, misleading, score interpr. 
Design measures, attainment of cognitive objectives 
Construct good essay questions 

Discuss features, uses, personality and interest inventories 
Compare advantages, disadvantages, different types measures 
Evaluate standardized tests, own subject, grade level taught 
Employ behavioral objectives 

Design measures, attainment of affective objectives 
Construct good objective items 

Discuss use of correlation in estimating test usefulness 
Contrast characteristics, standardized and teacher-made tests 
Evaluate uses of tests for placement and grouping 
Define own position, use of criterion- & norm-referenced tests 
Evaluate uses of tests in grading 

Employ methods of item analysis to improve objective items 
Compare features, rationales, types intelligence tests 
Describe origins, testing practice, why testing takes present forms 
Construct good anecdotal records 

Describe char ac teri s tic Sj uses, several t>^:)es norms 
Outline features, purposes, school and district testing programs 

Discuss Use, criterion-referenced tests 
Construct good semi -objective items 

Locate reviews, information, standardized tests, standard rcf. sourc< 
Characterize, interpret scores, measures of average, variation 
Discuss use, norm-referenced tests 

Use frequency count, histogram, to visualize & interpret sccros 
Examine relationship, two measures, using scatterplot 
Estimate relationship, two measures, using correlation coefficient 
Construct good rating scales, behavior checklists 
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Va.- '-.n )x Factor Solution for Importance Ratings 
oL 47 Measurement Objectives (N=234) 








Items 


Factor Loadings 

" I 11 III IV V VI VII VIII IX X XI XII XIII 
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52. Test uses, graditig 71 

42. Average, variation 59 

11. Objectives, planning 56 
47. Rating scales 56 
41. References, tests 37 

27. Objective items 51 

30. Test uses, plcmt, gpg. 49 38 
40. Serai -o b jec tivc items 48 

43. Use, norm-re£. tests 46 

17. Admin., scoring 44 

24. Stand, tests, own subj. 38 32 
10. Test uses, progress 3 5 

45. Scatterplot 63 

44. Hi'")Coyram 51 

46. Correl'n two meas. 47 

37. T\^3es, uses norms 39 

29. Stand., tchr, - tests 49 31 

34. Intelligence rests 35 

35. Origins testing 60 

14. Systems grading 57 
16. Ethical responsibility 56 

2. Tchr-stu. objectives 36 

38. District testing 38 47 

3. "Good", "bad" testing 43 

5. Test uses, impr. C&I 

6. Meas. what supp. to 

4. New t>-pes measures 31 
13. Adv., disadv.; dif f • meas. 32 37 

1. Testing enjoyable 

21. Essay questions 
33. Item analysis 

25. Behavioral objectives 

12. Own use tchr. -tests 

36. Anecdotr.l records (20) 

31. Grit. & norm-rof. tests 

39. Uses crit-ref. tests 

28. Gorrcl'n, test value 
9. Test, consistent job 

18. Estimate error 

i 

7. Own use, stand, test i 

22. Personality, interest | 

15. Cognitive objectives 

13. Dint, levels, objectives 

16. Measure affective 
20. Measure cognitive 
18. Affective objectives 

9. V/here norms useful 38 36 



35 
41 



36 



31 
32 



41 



33 



37 



66 
48 
40 

37 



32 33 



67 
53 
46 



65 

37 
(26) 



31 



6 
4 
4 
4 
3 
5 
4 
5 
4 
6 
4 
2 

5 

3 
4 
5 
3 

5 

3 

5 
t\ 

5 

5 
5 

6 

i\ 
I 

k 

1 



11 
54 



55 
51 
37 



65 
42 



49 



34 34 



58 
53 



54 

41 

35 



L 
L 



42 



ERIC 



Note. --Fac tor loadings >. 30 arc included in the table. Decimal points have 
been omitted. 



